Method and system of obtaining contact information for a person or an entity

ABSTRACT

A method of obtaining contact information for a person or an entity is disclosed. The method comprises detecting a name of the person or the entity in a frame from a screen of a device; determining a presence status for the detected name based on one or more connectivity status for the detected name in one or more communication applications and/or one or more databases, where the one or more communication applications operate on the device from which the frame originates and where the one or more databases are accessible on the device; displaying the presence status on the screen of the device.

CLAIM OF PRIORITY

This patent application is a continuation of U.S. patent applicationSer. No. 15/425,643, filed on Feb. 6, 2017, which is a continuation ofU.S. patent application Ser. No. 14/874,964, filed on Oct. 5, 2015,which is a continuation of U.S. patent application Ser. No. 14/628,907,filed on Feb. 23, 2015, which is a continuation of U.S. patentapplication Ser. No. 13/649,407, filed on Oct. 11, 2012, which in turnclaims, pursuant to 35 U.S.C. § 119, the filing date benefit of andright of priority to European (EP) Patent Application Serial No.11186621.6, filed on Oct. 26, 2011.

Each of the above identified applications is hereby incorporated hereinby reference in its entirety.

FIELD OF THE INVENTION

The present invention concerns a method and system of obtaining contactinformation for a person or an entity.

BACKGROUND OF THE INVENTION

Network applications are known in the prior art that provide access toan online network and having a number of users currently connected tosaid online network. The status of each user is shown on the computer ofa user, so it is possible to see if and how one of the other users canbe contacted. As an example, they can be contacted by a VoIP-call, anInstant Message, and via Email. Sometimes, the online status and/orlocation of the connected user are shown. As an example for such networkapplications MSN/Messenger network, AIM/Yahoo, Gchat/Google, WhatsApp,Facebook, Skype, Cisco and others can be cited. Currently, a user mustload these applications into his personal device, start theseapplications and enter names into it to see if someone's online, whattheir status is, how they can be contacted, etc. Furthermore, additionalsoftware like Pidgin or Trillian can be loaded, which show the presencein a variety of different applications.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of such systems with some aspects of the present invention asset forth in the remainder of the present application with reference tothe drawings.

BRIEF SUMMARY OF THE INVENTION

A system and/or method is provided for obtaining contact information fora person or an entity, substantially as shown in and/or described inconnection with at least one of the figures, as set forth morecompletely in the claims.

These and other advantages, aspects and novel features of the presentinvention, as well as details of an illustrated embodiment thereof, willbe more fully understood from the following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Various features and advantages of the invention will become apparentfrom the following description of non-limiting exemplary embodiments,with reference to the appended drawings, in which:

FIG. 1 illustrates an exemplary screenshot of a user's device.

FIG. 2 illustrates an exemplary screenshot, where different names havebeen recognized and marked, in accordance with an exemplary embodimentof the invention.

FIG. 3 illustrates an exemplary contact card of a recognized person withthe possibility of contacting the person via email, Instant messenger orcall.

FIG. 4 illustrates an exemplary list of all recognized names on a user'sscreen, including the presence in online networks applications.

FIG. 5 illustrates an exemplary contact card of a recognized person,where different persons with the same name have been found on theInternet.

DETAILED DESCRIPTION OF THE INVENTION

Certain embodiments of the present invention may be found in a methodand system for obtaining contact information for a person or an entity,as described in the following in more detail with reference to theattached figures.

As utilized herein, the term “exemplary” means serving as a non-limitingexample, instance, or illustration.

Aspects of the present invention enable presence and status control, andcontrol of how a person that is connected to an online network can bereached across a computer desktop including in applications andservices, without having to customize each application or service.Aspects of the present invention enable one set of presence controls ofa person connected to an online network that work for allpresence-providing services without the user having to think about whichservice may be connected.

According to an exemplary embodiment of the present invention, a methodof obtaining contact information for a person or an entity is provided,wherein the method comprises: detecting a name of the person or theentity in a frame from a screen of a device; determining a presencestatus for the detected name based on one or more connectivity statusfor the detected name in one or more communication applications and/orone or more databases, where the one or more communication applicationsoperate on the device from which the frame originates and where the oneor more databases are accessible on the device; and displaying thepresence status on the screen of the device.

Aspects of the present invention may enable, for example, detecting apresence of one or more persons or entities in applications running on adevice (e.g., a personal computer or a mobile phone) of a user,displayed on a screen of the device. This is done by a client sideapplication, which may run on a computer or mobile phone. In accordancewith various embodiments of the invention, the application may runelsewhere and communicate with the user's personal computer or mobilephone via, for example, the Internet. The applications may provideaccess to one or more online networks having a number of users currentlyconnected thereto. Examples for such network or communicationapplications may be MSN/Messenger network, AIM/Yahoo, Trillian, Pidgin,Gchat/Google, WhatsApp, Facebook, Skype, Cisco, etc. In theseapplications, a service is offered to show the presence of persons orentities, which are in the personal contact list of the user. A personor entity who is online can be contacted in various ways by the userincluding placing a VoIP call and sending an instant message.

In an embodiment, a screen shot of the screen is taken as a frame andthe names of all persons on the screen shot are recognized by, forexample, text recognition (e.g., Optical Character Recognition, or‘OCR’) program and thereafter a Named Entity Recognition (NER) algorithmfor finding names is applied. In an exemplary embodiment, a name of aperson on the screen may be selected manually by the user. In anotherexemplary embodiment, the screen shot may be sent to an internet-basedOCR and NER service for recognizing the person mentioned on the screenand sending back the detected name(s) to the personal computer or themobile phone. Alternatively, only one of the previously mentioned stepscould be done as an internet-based method, and the other may beperformed locally on the personal computer or mobile phone.

In an exemplary embodiment, recognizing presence-providing applicationsmay run on a mobile phone, and may be used in identifying in which ofthe applications the recognized person is currently connected couldadvantageously comprise checking a location-based internet service or aweb-service for an application. Recognizing at least one name of aperson may comprise searching a personal list of names of the user inthe computer or a mobile phone or contacting an internet data source.Displaying the presence status on the screen may comprise a presencelist, where the presence status of a number of persons may be displayed.

The marking of the recognized name may be done in a number of ways. Inone embodiment, a dashed-underline may be put underneath all recognizedpersons on the screen together with an icon. When the mouse hovers overthe name of the person or the icon, the details of the person aredisplayed. If the coordinates of the name in the picture or screen shotare known and a space is detected next to the name, the presence statusmay be displayed in the free space. Alternatively, the presence detailsof recognized names may be overlaid on top of the screen using atransparent window, which does not change or modify the underlyingapplication(s). However, presence information may be, for example,displayed in a corner of the screen. If available, additionalinformation for the person on the presence status can be shown.

In an exemplary embodiment, the presence of the person in theapplications indicated in the presence list may be updated continuouslyor periodically, so that the user may know of any change (e.g., when aperson goes offline in an application, changes status or location). Thepresence status gives the user the possibility to see how a person canbe contacted over a respective online network. Especially a VoIP-callcan be initiated or an instant message may be sent. In another exemplaryembodiment, a second one of the one or more communication applicationsis selected and a second communication session is initiated if thecommunication session cannot be established over a first one of the oneor more communication applications. Such a selection may be performedaccording to numerous criteria, including, but not limited to, personalpreference, communication quality, or cost.

In an exemplary embodiment, aspects of the invention relate as well to acomputer program product comprising a software code of a client sideapplication which can be loaded in a memory of a personal computer ormobile phone, and executed by a processor so as to run, when startingthe software code, a method of obtaining contact information for aperson or an entity, wherein the method comprises: detecting a name ofthe person or the entity in a frame from a screen of a device;determining a presence status for the detected name based on one or moreconnectivity status for the detected name in one or more communicationapplications and/or one or more databases, where the one or morecommunication applications operate on the device from which the frameoriginates and where the one or more databases are accessible on thedevice; and displaying the presence status on the screen of the device.

FIG. 1 illustrates an exemplary screenshot of a user's device. Referringto FIG. 1, there is shown a screen snapshot (1), which may correspond toa snapshot or any other frame of a screen of a user's device. In thisregard, a first step may comprise taking the screen snapshot (1) of theuser's device. The user's device may comprise, for example, a personalcomputer, a mobile phone, or any other device; and may comprise anysuitable logic, circuitry, interfaces, and/or code which may be operableto implement various aspects of the present disclosure.

The screen snapshot (1) may comprise, as shown in FIG. 1 for example, anormal text element shown on the screen. As understood, the method(s)described herein may be applicable to any application shown on screen,including, for example, presence applications (e.g., Facebook or Skype).The result may be that the presence of a person or entity in a singleapplication can be checked in other applications running on the personalcomputer or mobile phone. Some applications may be internet-based e.g.,Google or Facebook and the presence of a person or an entity can easilybe checked over the Internet. The internet based applications then caneasily be included into the presence control.

Normal personal computers have a built-in application to produce such ascreenshot or frames. From the frame of the screen text recognition isdone. This may be done by any known optical character recognition (e.g.,Optical Character Recognition, or OCR) software. When doing OCR of thesnapshot, the X/Y coordinates of the text are preserved so presencecontrols can be rendered in the right locations in the text.

Thereafter, names in the text are found using any known Named EntityRecognition (NER) software. This may involve determining a language ofthe text first. All online network applications running on the computeror a mobile phone might be identified. Once names and availablecommunication applications are known, in another step it is identifiedin which application the recognized person is currently connected. Then,a presence status is shown on the screen in which application therecognized person is currently connected. The presence status may give ageneral overview of, for example, persons, status, location, and/orpossibilities to contact the person, for any online network application.The step of displaying this presence status may comprise displaying apresence list, where the presence status of a number of persons whichhave been detected is displayed.

The OCR and/or the NER steps may be performed locally on a clientcomputer, or, they could be provided via an internet-based service,i.e., meaning the screenshot would be sent over the internet to a serverwhich would do some or all of the OCR and NER tasks and return theresults to the computer or mobile phone of the client. Still, it wouldbe possible that only one of OCR and NER may be done locally and theother is done as an internet based service. Alternatively, the usercould select manually a name of a person on the screen by the user withhis mouse. This selected name is copied to a clipboard, and thereafterthe name in the clipboard is looked up. Highlighting and rendering thepresence in the found applications may be done as described in relationwith FIG. 2, for example.

FIG. 2 illustrates an exemplary screenshot, where different names havebeen recognized and marked, in accordance with an embodiment of theinvention. In this regard, FIG. 2 shows the screen snapshot (1) of FIG.1, where different names have been recognized and marked. In thisexemplary embodiment, the names are underlined and beside the name anicon (2), which may be actionable (e.g., allowing for user input bayinteracting therewith), may be displayed and shown in which applicationthe person is present and how he or she can be reached. The client sideapplication may detect if a given contact has limited modalities throughwhich they can be reached. For example, in instances where anapplication running on the system (e.g., Facebook) may receive instantmessages, then if a contact is online, the software may indicate theavailable communication modalities as instant messages. The presencestatus of the person can as well be shown when, for example, the mouse(pointer) hovers over one of the names. An example of a contact card,which appears once the user chooses a person, is shown in FIG. 3.

FIG. 3 illustrates an exemplary contact card of a recognized person withthe possibility of contacting the person via email, Instant messenger orcall. Referring to FIG. 3, there is shown a contact card (3), which mayappear when a user chooses a person, as described, for example, withregard to FIG. 2. For example, when the icon (2), of FIG. 2, for aparticular person (“Eckard Laurien”) has been chosen, the contact card(3) shown in FIG. 3 may be displayed. The contact card (3) may indicate,for example, when the corresponding person is online on particularapplications (e.g., Skype and Google), and/or may also comprise contacticons (4), which may indicate the means by which the person may bereached (e.g., Email, Instant Messaging or IM, and/or a VoIP call). Theuser can directly click on one of the icons to contact directly theperson. In the embodiment of FIGS. 2 and 3, the presence control processmay be overlaid on the text parts of the screen or on the runningapplications without modifying the state of the applications.Alternatively, a space can be detected next to a recognized name andshowing the presence there. Of course, any additional information forthe person can be shown on the presence list.

In some instances, if the coordinates (e.g., X/Y) of a screen shot areunavailable, a small window of contacts (e.g., contact lists) may beshown somewhere on the screen, e.g., in a corner. An example of such acontact lists is shown in FIG. 4.

FIG. 4 illustrates an exemplary list of all recognized names on theuser's screen, including the presence in online networks applications.Referring to FIG. 4, there is shown a contact lists (5). In this regard,the contact lists (5) may be utilized in instances where, for example,the (X/Y) coordinates of a screen shot maybe unavailable, thusnecessitating use of other means to display contacts relatedinformation—e.g., a small window of contacts may be shown somewhere onthe screen, e.g., in a corner. The contact lists (5) may display, foreach recognized contact, the communication application(s) by which thecontact may be online and the contact icon(s) (4) indicating one or moremodes of reaching the contact.

Both the contact card (3) of FIG. 3 and the contact lists (5) of FIG. 4,once shown on the screen (of user's device), may be continuously orperiodically updated, for example, as the presence of the person in theapplications might change over time. Therefore, it is possible that if aperson is not contactable in a way through a specific application,although shown in the list, the client side application automaticallychooses a different application, to which the recognized person iscurrently connected. This process may be transparent to the user.

Additionally or alternatively, if Named Entity Recognition (NER) doesnot work, an alternative search may be performed—e.g., using textrecognition (OCR) program for names against a personal list of names ofthe user stored in the computer or a mobile phone.

Alternatively, the names recognition may comprise contacting an internetpage or internet data source, such as a telephone book or any otherregister, where names may be found. FIG. 5 shows an example of contactlist with information from an Internet page.

FIG. 5 illustrates an exemplary contact card of a recognized person,where different persons with the same name have been found on theInternet. Referring to FIG. 5, there is shown a contact list (6) withinformation from an Internet page XY. In this case, as different optionsare available, these options may include found telephone numbers beingshown, such as when clicking on or hovering over the name or icon (2) inFIG. 2 for example.

Furthermore, there are location-based internet services like FourSquarethat allow people to “check in” on the internet, which makes theirlocation known. In the framework of the invention, such location-basedinternet services could be used to show that the person is online,available, checked into Desperados Restaurant 9 minutes ago, forexample. Lync (Microsoft Communicator) has as well automatic locationservices built in, which could be used. A graphical aid could be used,for example, the closer someone is to the location of the user, and thenthe larger the presence icon would become, so at a glance the user couldsee who is geographically close to them.

Furthermore, it could be possible that an online calendar of arecognized person is used to show what a person is doing at the moment(e.g., “available, online, has meeting ABCD in 10 minutes”). A graphicalaid could be used, e.g., contacts that are about to become unavailablebecause they have a meeting soon could have their name in italics, orhave a partially faded presence icon, etc.

Other embodiments of the invention may provide a non-transitory computerreadable medium and/or storage medium, and/or a non-transitory machinereadable medium and/or storage medium, having stored thereon, a machinecode and/or a computer program having at least one code sectionexecutable by a machine and/or a computer, thereby causing the machineand/or computer to perform the steps as described herein.

In accordance with various embodiments of the invention, the user maystart the client side application and any related step of the inventivemethod by pressing a key or a combination of keys of a keyboard of thepersonal computer or mobile phone. In an exemplary embodiment, aspectsof the invention relate also to a computer program product comprising asoftware code of a client side application which can be loaded in amemory of a personal computer or mobile phone, and executed by aprocessor so as to run the steps of any of the inventive claims. In anexemplary embodiment, aspects of the invention relate as well to acomputer program product comprising a software code of a client sideapplication which can be loaded in the memory of a personal computer ormobile phone, and executed by a processor so as to run the steps of anyof the claims. The following three examples are given as example,non-limiting embodiments.

Example 1

Calling from Facebook. User is connected to Facebook, and sees from astatus update that a friend is back from vacation. User uses aspects ofthe disclosure to see the presence of the contact in availablecommunication applications. User interacts with the control to place acall over the online network that the friend is logged into, forexample.

Example 2

Chatting with WhatsApp users from Skype. User has Skype open on theircomputer, wants to chat with a friend, but the friend is offline inSkype. So user uses aspects of the disclosure to determine the presenceof the contact, and sees they are online in WhatsApp. User interactswith the presence control and starts an Instant Message IM conversationover WhatsApp.

Example 3

Internet page integration. User viewing a PDF, sees a name and wants tocall that person. User uses aspects of the disclosure to determine thepresence of the contact. User interacts with the control and sees thatalthough the contact is not on the user's buddy list, the contactdetails were found on an internet page. User uses the control to place acall.

Other embodiments of the invention may provide a non-transitory computerreadable medium and/or storage medium, and/or a non-transitory machinereadable medium and/or storage medium, having stored thereon, a machinecode and/or a computer program having at least one code sectionexecutable by a machine and/or a computer, thereby causing the machineand/or computer to perform the steps as described herein.

Accordingly, the present invention may be realized in hardware,software, or a combination of hardware and software. The presentinvention may be realized in a centralized fashion in at least onecomputer system, or in a distributed fashion where different units arespread across several interconnected computer systems. Any kind ofcomputer system or other apparatus adapted for carrying out the methodsdescribed herein is suited. A typical combination of hardware andsoftware may be a general-purpose computer system with a computerprogram that, when being loaded and executed, controls the computersystem such that it carries out the methods described herein.

The present invention may also be embedded in a computer programproduct, which comprises all the features enabling the implementation ofthe methods described herein, and which when loaded in a computer systemis able to carry out these methods. Computer program in the presentcontext means any expression, in any language, code or notation, of aset of instructions intended to cause a system having an informationprocessing capability to perform a particular function either directlyor after either or both of the following: a) conversion to anotherlanguage, code or notation; b) reproduction in a different materialform.

While the present invention has been described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted withoutdeparting from the scope of the present invention. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the present invention without departing from its scope.Therefore, it is intended that the present invention not be limited tothe particular embodiment disclosed, but that the present invention willinclude all embodiments falling within the scope of the appended claims.

What is claimed is: 1-16. (canceled)
 17. A method comprising: performingrecognition on content captured from a display of a mobile device;identifying an entity in the content; issuing a search for the entity;extracting contact information for the entity from a data source;generating a contact card for the entity, wherein an action associatedwith the contact card for the entity is generated from the contactinformation extracted from the data source; and providing the contactcard for display to a user of the mobile device, wherein the mobiledevice displays the contact card.
 18. The method of claim 17, whereinissuing a search for the entity comprises: determining that the firstentity is classified as a name; searching a contact list of the user ofthe mobile device using the entity; and returning information from thecontact list as search results for the search when the entitycorresponds to a contact in the contact list.
 19. The method of claim18, wherein issuing the search for the entity comprises: searching anInternet data source when the entity fails to correspond to a contact inthe contact list.
 20. The method of claim 17, wherein the action for theentity is a communication modality.
 21. The method of claim 17, whereinthe search results comprise a graphical aid that indicates a relativeproximity to the entity.
 22. The method of claim 17, wherein generatingthe contact card for the entity comprises: identifying a service for theentity; determining that service corresponds to a mobile applicationinstalled on the mobile device; and displaying an icon associated withthe action, wherein the action opens the mobile application.
 23. Themethod of claim 17, wherein the entity corresponds to a contact in acontact list and generating the contact card comprises: determining acommunication modality for the entity according to an application on themobile device; and generating the contact card using information from acontact list that corresponds to the communication modality.
 24. Asystem comprising a software code of a client side application which canbe loaded in a memory of a mobile device, and executed by a processor toperform operations comprising: performing recognition on contentcaptured from a display of the mobile device; identifying a first entityin the content; issuing a search for the first entity; extractingcontact information for the first entity from a location-based Internetservice; generating a first contact card for the first entity, whereinan action associated with the first contact card for the first entity isgenerated from the contact information extracted from the location-basedInternet service; and providing the first contact card for display to auser of the mobile device, wherein the mobile device displays the firstcontact card, and wherein a graphical aid associated with the display tothe user is indicative of a distance between the user and the firstentity.
 25. The system of claim 24, wherein the operations comprise:identifying a second entity in the content; and generating a secondcontact card for the second entity.
 26. The system of claim 25, whereinthe operations comprise: displaying that the user is geographicallycloser to one of the first entity and the second entity.
 27. A systemcomprising a software code of a client side application which can beloaded in a memory of a mobile device, and executed by a processor toperform operations comprising: performing recognition on contentcaptured from a display of the mobile device; identifying a first entityin the content; issuing a search for the first entity in a contact listfor a user of the mobile device; extracting contact information for thefirst entity from the contact list; accessing presence information forthe first entity from a presence-providing application running on themobile device; generating a first contact card for the first entity,wherein a first action associated with the first contact card for thefirst entity is generated according to the contact information and thepresence information; and providing the first contact card for displayto the user of the mobile device, wherein the mobile device displays thefirst contact card.
 28. The system of claim 27, wherein the operationscomprise: accessing an online calendar of the first entity; anddetermining the presence information according to the availability ofthe first entity.
 29. The system of claim 27, wherein the operationscomprise: displaying the first contact card is in a positioncorresponding to a location of the first entity in the content.
 30. Thesystem of claim 27, wherein the operations comprise: identifying asecond entity in the content; and generating a second contact card forthe second entity.
 31. The system of claim 27, wherein the first actioninitiates a first mobile application, and wherein a second actionassociated with the first contact card for the first entity initiates asecond mobile application.
 32. The system of claim 31, wherein thememory stores instructions that, when executed by the processor, causethe mobile device to: receive a selection of the first action; andlaunch the first mobile application using the contact information. 33.The system of claim 27, wherein performing recognition on the contentdisplayed on the display includes: examining accessibility datagenerated for the content displayed on the display.
 34. The system ofclaim 27, wherein identifying the first entity includes using a nameclassifier to determine a set of words that are classified as a name.35. The system of claim 27, wherein the presence information for thefirst entity is periodically updated over time.
 36. The system of claim27, wherein the contact list for the user comprises information from aremote Internet page.